Skip to content

Conversation

pbalcer
Copy link
Contributor

@pbalcer pbalcer commented Apr 14, 2025

No description provided.

@pbalcer pbalcer requested a review from a team as a code owner April 14, 2025 08:54
@pbalcer
Copy link
Contributor Author

pbalcer commented Apr 14, 2025

There are a few things I noticed when writing this that I think we should fix:


* `passed`: Boolean indicating if verification passed (default: `True`).

We should just throw exceptions when a benchmark fails, not produce an invalid result.


* `name`: Set to `label` by the framework.

The name field in Result is inconsistently used throughout the framework. We should clean this up and always use label where appropriate.


* **Use unique names:** Ensure `benchmark.name()` and `result.label` are descriptive and unique.

What I was contemplating writing is that benchmark names should ideally never change because they are unique keys the visualization scripts use to collect benchmark data into series.
But this is a very annoying limitation. We should introduce a (maybe autogenerated?) key to groups/benchmarks that is unique and not used for any sort of visualization (or maybe treat the existing name/label as a key, and add a new "display name"?)

@PatKamin
Copy link
Contributor

What I was contemplating writing is that benchmark names should ideally never change because they are unique keys the visualization scripts use to collect benchmark data into series. But this is a very annoying limitation. We should introduce a (maybe autogenerated?) key to groups/benchmarks that is unique and not used for any sort of visualization (or maybe treat the existing name/label as a key, and add a new "display name"?)

I think that new "display_name" and "display_label" variables would work best in this situation. They could by default take the current value of name/label and be overridden, if needed.

@pbalcer
Copy link
Contributor Author

pbalcer commented May 14, 2025

@intel/llvm-gatekeepers please merge

@sarnex sarnex merged commit 6c4f597 into intel:sycl May 14, 2025
25 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants